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Abstract 

An accurate knowledge of the fluorescence yield and its dependence on atmospheric 
properties such as pressure, temperature or humidity is essential to obtain a reliable 
measurement of the primary energy of cosmic rays in experiments using the fluorescence 
technique. In this work, several sets of fluorescence yield data (i.e. absolute value and 
quenching parameters) are described and compared. A simple procedure to study the 
effect of the assumed fluorescence yield on the reconstructed shower parameters (energy 
and shower maximum depth) as a function of the primary features has been developed. 
As an application, the effect of water vapor and temperature dependence of the collisional 
cross section on the fluorescence yield and its impact on the reconstruction of primary 
energy and shower maximum depth has been studied. 



1. Introduction 

Fluorescence telescopes record the longitudinal profile of extensive air showers induced 
by very energetic cosmic rays through the detection of the fluorescence light generated 
by secondary charged particles. This technique allows an accurate determination of the 
shower maximum depth A max . In addition, since the fluorescence intensity is propor- 
tional to the deposited energy, the integration in depth of the fluorescence profile allows 
a calorimetric determination of the shower energy. The total primary energy is later 
obtained by applying a correction accounting for the so-called missing energy. A key pa- 
rameter for the reconstruction of the calorimetric energy is the fluorescence yield (FY), 
defined as the number of fluorescence photons emitted per unit deposited energy. The 
FY which depends on the atmospheric parameters (i.e. pressure P, temperature T, hu- 
midity) is measured in dedicated laboratory experiments. Presently one of the largest 
contributions to the total systematic error in the primary energy comes from the ab- 
solute value of this parameter. Also the uncertainties in the dependence of the FY on 
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atmospheric parameters might have a non-negligible effect on both primary energy and 
I max measurements. 

In this paper a simple procedure to study the effect of the assumed fluorescence 
yield on the reconstructed shower parameters will be shown. Preliminary results were 
published in pQ. In section [2] a brief summary of the FY properties will be presented. In 
the next section the features of some data sets of FY commonly used will be described 
and compared. A simple method to evaluate the effect of the FY selection on shower 
reconstruction is shown in section 4. The shower development will be described by a 
Gaisser-Hillas profile. The effect of a variation in the FY (including its atmospheric 
dependence) is a change in the reconstructed longitudinal development of the deposited 
energy which can be easily evaluated. Results for the dependence of both reconstructed 
I max and primary energy on the FY selection will be presented in section [5j 

Keilhauer and Unger [5] have carried out a similar study focused on estimating the 
influence of the atmospheric conditions on the reconstructed shower parameters. However 
our approach is somewhat different since we only need the Gaisser-Hillas profile, while 
in [2] simulated showers are used for the analysis. 



2. Air-Fluorescence Yield 



Air-fluorescence in the near UV range is basically produced by the de-excitation 
of atmospheric nitrogen molecules excited by the shower electrons. The spectrum of 
fluorescence consists of a set of molecular bands represented by their wavelengths A and 
therefore the total fluorescence yield Y in a given wavelength interval AA can be obtained 
by adding up the contributions of all the molecular bands Y\. 

y(x) = £y a (x). (i) 

AA 



Excited molecules can also decay by collisions with an environmental molecule. Because 
of this effect, the FY in the absence of quenching Y? (that is, at null pressure) is reduced 
by the so-called Stern- Volmer factor. 

Y ^ = i + P?nx,T) - v 

The dependence of Y\ on atmospheric conditions can be described by a single parameter, 
the characteristic pressure P' which is defined as the pressure for which both radiative and 
collisional de-excitation have the same probability. In general P' contains a contribution 
of all possible quenchers i (i.e., N2, O2, H2O). 

1 fi p , kT 1 _ f8kT 

P i P i T CT NifNi V TT^Ni 

In the above expressions /, is the fraction of molecules of type i in the mixture, ctni is 
the collisional cross section which depends on the particular band, and vni and //jvi are 
the relative velocity and reduced mass of the two body system N-i respectively; k is the 
Boltzman constant and r the radiative lifetime of the corresponding level. 
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A number of experimental results of the FY in dry air are available (a review is pre- 
sented in [3] and a critical comparison of the absolute measurements can be found in 
jl]). Usually the absolute values of Y\ are measured at atmospheric pressure and room 
temperature and the P' parameters are evaluated at the same temperature. Equation ^ 
allows the determination of Y® and the dependence of the FY with pressure. Unfortu- 
nately measurements of the P' parameters show large discrepancies and therefore the Y® 
parameter is not suitable for comparison between different experiments. 

According to expression j3j|, the P' value for dry air can be expressed as 

1 = /o + /n (4) 

PL(^T) P^\,T)^ ift(A,T)' U 

where /o = 0.21 and /n = 0.79 are the fraction of oxygen and nitrogen molecules in air 
respectively. Measurements of Pq(X) and P^(X) are also available [5]. 

2.1. Humidity dependence 

Water molecules are also effective quenchers for excited N2 and therefore fluorescence 
emission is partly suppressed in humid air. The characteristic pressure for humid air 
Pj' lum (A,T) can be calculated from equation ^ by including the water contribution. 
The relationship between the characteristic pressures of dry P^ (X,T) and humid air 

1 ~Sr +T¥7TS. ( 5 ) 



KuJ^T) P^ y (X,T)\ P) pf;(a,t) 

where P w is the partial pressure of water and P^ is the characteristic pressure for fluo- 
rescence quenching by collisions with water molecules. 

Measurements of P„ values have been performed by several authors. The AIRFLY 
collaboration has published results for four bands at 293 K [B]. In ref. [5] experimental 
values of P^ for three relevant bands are also reported. More details can be found in [3] . 

2.2. Temperature dependence 

The T dependence of the fluorescence yield is given by equations ^ and ([3| . Neglect- 
ing the dependence of the collisional cross section on the kinetic energy of the colliders, 
P' is proportional to vT- Assuming a measurement of the characteristic pressure at a 
reference temperature T , the T dependence of P/(A,T) is given by 

P((\,T)=Pl(\,T )^r. (6) 

However, as it is well known, the average collisional cross section follows a behavior which 
can be described by a power law (~ T a ) [7]. Therefore equation ^ becomes 



The dependence of the FY with both T and P can be easily predicted by combining 
equations ([2| , ^ and J7| . The temperature dependence of the collisional cross section for 

3 



dry air has been measured recently by the AIRFLY collaboration and the corresponding 
values of a\ parameter for four molecular bands have been published [6]. In principle, 
this parameter depends on the quencher (nitrogen, oxygen, water). However independent 
measurements of a\ for each air component is difficult and usually it is assumed that 
a\ takes the same value for all components. On the other hand ct\ values for water 
are not presently available. However, as we will show below all these uncertainties and 
approximations have a marginal impact on the shower reconstruction. 

3. Air-fluorescence yield data 

3.1. Required information 

The reconstruction of the shower parameters requires the following information on 
the fluorescence yield. 

1) The absolute value in dry air at a given pressure and temperature for all bands 
within the spectral range of the telescope. As an alternative, the absolute value for 
a reference transition (e.g. 337 nm) and the relative intensities of the bands at a 
given pressure will also provide the total fluorescence yield. 

2) The values of P' in dry air at a reference temperature for all bands. Measurements 
of P^(X,T ) and Pq(X,T ) could be also used for this purpose. 

3) The values of P^(X,To) for all wavelengths. 

4) The T dependence of oni for all wavelengths. In principle a\ values for each 
quencher would be needed, however this information is, at present, rather limited. 

A data set containing 1) and 2) allows us to determine the total FY for dry air as a 
function of P at the reference temperature Tq. Neglecting the T dependence of <?Ni, the 
total FY can be calculated at any P and T condition. Adding 3), the air-fluorescence 
yield can be evaluated for humid air. Finally, 4) provides a more accurate extrapolation 
at temperatures far from the reference one (i.e. at high altitude). 

Apart from pioneering works jS], in the last 10 years several measurements of the FY 
have been carried out in laboratory experiments injecting accelerated electrons into air 
targets [3] . Nowadays three data sets combining some of these measurements are mainly 
being used in cosmic ray experiments using fluorescence telescopes. They are those of 
Kakimoto-Bunner, Nagano and the combination Nagano-AIRFLY. These data sets are 
described below. In all of them the humidity effect is neglected and the T dependence of 
quenching is calculated assuming a constant collisional cross section. The effect of these 
approximations on the fluorescence yield is not negligible and will be also evaluated in 
this section. 

3.2. Kakimoto-Bunner (K-B) 

This FY data set which was used by the HiRes collaboration in 2001 [9] is a combina- 
tion of fluorescence yield measurements of Kakimoto et al. |10j and the relative intensities 
of the molecular bands reported by Bunner . 

Kakimoto et al. measured the absolute value of the air-fluorescence yield expressed 
in photons per meter e\ for the three primary molecular bands of nitrogen (337, 357 
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and 391 nm) at 1000 hPa and 288 K. The remaining fluorescence bands are found to 
contribute with about a 30% of the total fluorescence photons when the filter employed 
in the HiRes telescopes (bandpass filter between 295 and 405 nm) is considered. As 
pointed out by [5], these data translate into an absolute FY contribution of 0.46 ph/m 
for these bands. The remaining fluorescence spectrum (13 bands) is distributed according 
to the relative intensities collected by Bunner |11) . 

Kakimoto et al. calculated the fluorescence efficiency of the primary bands (defined 
as the fraction of deposited energy emitted as fluorescence radiation) <f>\ as 

where h is the Planck constant, c is the speed of light, (dE/dX) d is the energy deposited 
by the electron per unit of atmospheric depth, and p is the air density. 

Experimental results of pressure dependence for the primary bands are also reported 
by Kakimoto et al. In that work the FY is parameterized as 

e A (P,T) = (9) 

l + pB x VT 

In this expression A\ is the number of photons emitted per electron and unit column 
density (g/cm 2 ) in the absence of quenching (i.e. P=0) and the B\ parameter is related 
with the characteristic pressure through and Q by 



B ^wr <10) 

where i? gas is the specific gas constant. 

The implementation of <f)\ and B\ in equation Q can be easily performed. Firstly, 
the fluorescence yield is related with the fluorescence efficiency by 

Yx = K-±. (ID 

Therefore the YP values can be obtained from the fluorescence efficiency at Po=1000 hPa 
and the B\ parameters at To = 288 K from 

<l2) 

Finally, the values of P' x inferred from the reported B\ parameters of Kakimoto et al. 
for the 337 nm band are used for all 2P components while that of the 391 nm is used for 
the IN system. 

3.3. Nagano 

Nagano et al. P2, provided experimental results of e\ in dry air for 15 nitrogen bands 
at 1013 hPa and 293 K as well as the corresponding P' x values at the same temperature. 
Using these measurements, the authors calculate the fluorescence efficiency at null pres- 



sure 0^. The corresponding Y? values can be easily obtained applying expression (111. 
Notice that both Kakimoto et al. and Nagano et al. assume for this calculation that 
the energy lost by the electron is fully deposited in the field of view of the experimental 
set-up. Implication of this approximation has been discussed in [3], [T3] and |14j . 
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Figure 1: Ratio of Nagano and Kakimoto-Bunner fluorescence yields to that of Nagano- AIRFLY versus 
atmospheric depth. The ratio K-B/N-A of HY\(X)e\, also displayed, indicates that the effect of the 
optical efficiency (filter and telescope) reduces significantly the large disagreement in the total FY of 
both data sets. 



34. Nagano-AIRFLY (N-A) 

In this data set the absolute scale is given by the <fi® 37 value reported by Nagano et 
al. for the 337 nm band which translates into a Y337 value of 6.38 ph/MeV at Po=800 
hPa and To=293 K. Relative intensities I\ at To and Pq for 34 wavelengths and the 
corresponding P'(A) values in dry air at 293K reported by AIRFLY [TS] are used to get 
a complete set. The corresponding Y® for any wavelength is thus obtained as 

= WP>,To) • h(Po,T ) ■ (l + pl ^° To) ) . (13) 

This data set is presently used by the Pierre Auger Observatory [To] . 

3.5. Comparison of data sets 

The FY for the above described data sets has been compared. In figure [l] the ratios 
^Nagano/^N-A and Ik-b/^n-a against atmospheric depth are shown. The large devi- 
ations observed at low depth are due to discrepancies in the P' values in the data sets. 
Obviously these discrepancies have no relevant impact in shower reconstruction since in 
our energy range they take place before the development of the shower. On the other 
hand, at larger depths differences around 20% between N-A and K-B data sets are found 
while a relative small discrepancy of around 2% is observed between N-A and Nagano 
descriptions. 

In practice the comparison between FY data sets should include other ingredients. 
Fluorescence photons generated by cosmic ray showers are detected with ground tele- 
scopes located at a large distance from the emission point and thus suffering a non- 
negligible attenuation which is wavelength dependent. In addition, the optical systems 
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of the telescope, including the filter, the mirror and the PMT photocathode have a re- 
sponse which is also wavelength dependent. Therefore the relative differences between 
two FY data sets are not directly translated to the energy reconstruction since the relative 
intensities of the molecular bands are modified by the detector efficiency. For instance, 
discrepancies between two data sets which only affect wavelength regions for which the 
filter has a low transmittance might have a marginal impact on energy reconstruction 
although the discrepancy in total FY is large. The parameter more closely related with 
possible deviations in the shower reconstruction is the product Y\{X)e\T\(X , Xo) where 
E\ is the optical efficiency of the telescopes including all components and T\(X, Xq) 
represents the atmospheric transmittance between the emission point and the telescope 
location. 

The effect of the optical filter is particularly relevant in the comparison between 
K-B and N-A data sets. As shown in figure [T] the large disagreement between them 
nearly disappears when the efficiency of the optical system of the Auger telescopes [T7] 
is included. Figure [2] shows the fluorescence yield for both data sets (left) and the effect 
of the optical efficiency (right). Note that the large discrepancy in the bin centered at 
317 nm is smoothed by the strong absorption of the filter at this wavelength. However 
the discrepancies at wavelengths with good transmission, and thus less sensitive to the 
filter, nearly compensate each other. 

In a similar way, it has been shown |12) that the large disagreement between the 
Nagano and K-B data sets are significantly reduced when the optical efficiency of HiRes 
is included. 




X [nm] X [nm] 



Figure 2: Fluorescence yield in absolute units versus wavelength for the N-A and K-B data sets (left). 
Since K-B intensities are given in 9 nm bins [9], the N-A fluorescence yields have been recalculated for 
the same wavelength bins for a better comparison. The effect of the Auger optical efficiency (black line) 
on the FY of both data sets is illustrated (right). 



3.6. Effect of temperature and humidity on the FY 

The above described data sets do not include the contribution of quenching by water 
molecules in humid air. In addition, the temperature dependence of the characteristic 
pressure is assumed to follow a vT law (i.e. a\ is assumed to be zero). The effect of 
these approximations on the FY profile has been evaluated using the monthly average 
description for the atmosphere measured at the Auger site [18] . As an example, in figure[3] 
the effect of including the humidity and temperature corrections (December month) in the 
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Figure 3: Effect of the humidity and a T-dependent collisional cross section on the N-A data set. fn this 
plot the ratio of fluorescence yields for the month of December versus atmospheric depth is shown. 



N-A data set is shown against atmospheric depth. For these calculations the quenching 
parameters of [6] have been used. As expected, humidity decreases the fluorescence yield 
at large depth (i.e. near ground) while the effect of the T term is only relevant at high 
altitudes. Neglecting the possible dependence of the collisional cross section N2-H2O 
with T (i.e. assuming u\ is zero for water) , the total effect of humidity and temperature 
can be easily computed. The result is also shown in figure [3] A similar behavior has been 
found in a study carried out in |19j using several sets of fluorescence yield data including 
the temperature and humidity dependence of quenching from various authors. 

As expected, neither the optical efficiency of the detector nor the atmospheric trans- 
mission introduce any significant effect in these results since atmospheric conditions (i.e. 
temperature and humidity) has a negligible impact on the fluorescence spectrum. 



4. Method 

In this note we propose a simple procedure to evaluate the effect of the fluorescence 
yield selection on the reconstruction of shower parameters. Assuming a certain longitudi- 
nal profile of deposited energy dE(X)/dX, the number of fluorescence photons generated 
per unit atmospheric depth is determined by the fluorescence yield in the wavelength in- 
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terval of the telescope. 

< 14 > 

The total calorimetric energy E can be obtained from the integral of the longitudinal 
profile and therefore 

f°° 1 dn« cn , 

If the FY assumption is now changed to a new data set Y'(X), the profile of deposited 
energy consistent with the fluorescence profile is 

dE' dE Y[X) 

dX dX Y'(X) ' 1 ' 

and thus the calorimetric energy associated to the new FY selection will be 

E^r^^dX. (17) 
J dX Y'(X) V ' 

Therefore, in principle, the effect of changing the FY selection on the primary energy 



can be evaluated by comparing ( 15 ) and ( 17 1. On the other hand, the effect on the X 



L max 



value can be obtained by comparing the shape of the dE/dX and dE' /dX profiles. 

Only a small fraction of the fluorescence photons generated by the shower reaches the 
PMT camera of the fluorescence detector. In the first place, the number of photons is 
strongly reduced by a factor A/ (AttR 2 {X)) where A is the area of the telescope and R is 
the distance from the emission point to the telescope location. In addition, as previously 
pointed out, the atmosphere scatters a non- negligible fraction of photons on their way to 
the telescope. Finally the optical elements of the telescope also absorb a certain fraction 
of those photons reaching the telescope window. Both atmospheric transmission and 
optical efficiency are wavelength dependent. Therefore the profile of deposited energy is 
calculated from the profile of observed photons using the expression 



dE dn° hs 4ttR(X) 2 1 



dX dX A £ AA e x ■ T\(X) ■ Y\(X) ' 



and thus, relationship ( 16 ) becomes 

d&_ = dE J2Y x (X)-e x -T x (X) 
dX dX J2Y{(X)-e x -T x (X) 



(18) 



(19) 



Therefore the shower maximum depth has to be evaluated using ( 19 1 instead of ( 16 ) and 



the calorimetric energy E' inferred from the new FY selection is given by 

_ [°° dE J2Y x (X)-s x -T x (X) 
E -Jo dJcEY^X).e x .T xi X) dX - (20) 

The variation of both X max and E due to a change in the FY selection is not sensitive to 
the fine details of the longitudinal development of dE/dX and thus instead of using real 
data or simulated showers it is easier and more convenient for this study to describe the 
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longitudinal development of deposited energy by a Gaisser-Hillas (GH) profile [2U] given 

by 

dE /dE\ ( X-X - X ")/ A 



- 



dX \ dX J x \ X max — X, 



e—^- , (21) 



where Xq and A are shape parameters and Y max is the shower maximum depth. 

We have studied the impact of the FY selection for proton and iron showers with 
zenith angle 9 of 30° and 60° and fixed primary energies E of 10 19 and 10 20 eV. The 
corresponding calorimetric energies have been calculated assuming an invisible energy 
as parameterized by |21) . The values of Xq, A and X max have been obtained by fitting 



the average longitudinal development of a sample of CORSIKA showers [22] to eq. (21 ). 
These longitudinal developments are fully compatible with those shown in 23 . The 
value of (dE/dX) x is obtained as a normalization constant to account for the total 
deposited energy. 




Figure 4: Comparison of FY N-A and K-B descriptions and their impact in the shower reconstruction for 
a E = 10 19 eV proton shower with 30° zenith angle (black profile). The total FY of K-B and N-A (red 
axis on the right) are significantly different and therefore with a large impact on the profile of deposited 
energy (red profile). When including the optical efficiency of the telescopes the FY ratio (blue axis on 
the right) is very close to one and their impact on deposited energy in nearly negligible (blue profile). 



For a given energy, the shower reaches its maximum development at an altitude which 
grows with 9 and therefore, since the FY varies with altitude (through pressure, tempera- 
ture and humidity) , the impact of the FY selection depends on the shower geometry. The 

10 



effect of atmospheric transmission increases with the distance between the telescope and 
the shower axis. Values of 10 and 30 km have been tried for this parameter. As already 
mentioned the atmosphere has been modeled using monthly average profiles measured 
at the Auger site [IS] . 

As an example we show in figure[4]the effect in the profile of deposited energy of a 10 19 
eV proton shower when changing from Y^~ (X) to Y<~~ B (X) fluorescence yield. The 
black solid line is the GH profile for a shower with an E value of 10 19 eV. The red dashed 
line represents the ratio Y^~ B /Y^~ (red axis on the right) as a function of depth. 



Applying (16), the modified profile (i.e., the profile of deposited energy which would be 
obtained using the K-B fluorescence yield), has been calculated (red dotted line). The 
integral of this profile results in a total deposited energy significantly larger (by about 
a 20%). However when taking into account the effect of the optical efficiency, the ratio 
Y^~ B (X) -S\/J2 Y^~ A {X) ■ s\ is close to unity except at very low atmospheric depth 
(dashed blue line measured in the blue axis on the right) and therefore the reconstructed 



profile from expression ( 19 1 (dotted blue) is nearly unaffected giving a deviation in the 
deposited energy around 2% in full agreement with previous studies [24]. The effect 
of the atmospheric transmission does not introduce significant differences between FY 
data sets in this case. This example has been calculated using the atmospheric profile of 
December. Similar results have been found for other seasons. 
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Figure 5: Effect of including the humidity contribution in the profile of energy deposited for 30° (left) 
and 60° (right) zenith angles for a Fe shower of 10 20 eV. Notice that the dependence of the FY ratio 
with slant depth varies with the incoming angle. See text for details. 



The effect of humidity and that of a T-dependent collisional cross section has also 
been studied with this method. Some typical examples are shown in figures [5] and [6] 
In figure [5] the effect of water vapor (December month) in the N-A data set for a Fe 
shower of 10 20 eV and angles of 30° and 60° is studied. The development of the shower 
depends on the slant depth and therefore the smaller is 9, the closer to the ground the 
maximum shower development is reached. Since humidity is stronger at low altitudes, 
its effect is more important for vertical showers. In this case the calorimetric energy has 
to be increased by about a 4% at 30° while at 60° the correction is negligible (< 1%). 
On the contrary the effect of neglecting the T dependence of the collisional cross section 
is stronger at low temperatures (i.e. at high altitudes) since the T value at which the 
FY is measured are close to that at ground. This feature can be observed in figure [6] for 
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Figure 6: Effect of including the temperature dependence of the cross-section in the shower reconstruction 
for 30° (left) and 60° (right) zenith angles for the same case of figure [5] See text for details. 

the same kind of shower assuming the same atmospheric profile. In this case the effect 
of temperature can be corrected by increasing E by about 2% (6%) at 30° (60°). 

The corresponding effect of the correction on the shower maximum depth is also 
displayed in the above figures. As expected the A max value increases (decreases) if the 
slope of the FY ratio versus depth function at X = X max is negative (positive) . 

The longitudinal profiles have been calculated until the showers reaches ground level 
that in the case of the Auger site corresponds to a vertical atmospheric depth of about 
880 gem" 2 . 

5. Results 

The method described above has been used to predict the impact of various FY 
assumptions on the shower reconstruction. The relative difference in reconstructed pri- 
mary energy between several FY selections and the N-A one 8e = (Eq — Eq~ a )/ Eq~ a 
has been calculated as well as the absolute difference of the shower maximum depth 
AX max = X max — A,^ ax A . The effect of the optical efficiency of the telescope [T7] (in 
particular that of the filter) and the atmospheric absorption to these deviations has also 
been evaluated. 

As already mentioned the contribution of the invisible energy has been calculated 
following the parameterization of [21] . Since typical variations in deposited energy due 
to the FY selection are smaller than 10% and the invisible energy contributes an amount 
ranging from 8% and 12%, the corresponding deviations in Eq are smaller than the one 
of the calorimetric energy by as much as one percent unit. 

In the first place the FY data sets of K-B and Nagano have been compared with the 
N-A one. The results for total energy are displayed in tables^] (10 20 eV) and[2](10 19 eV) 
while those of X max are shown in tables [3] (10 20 eV) and[4](10 19 eV). As can be seen 
in tables [l] and [2j the Nagano FY would lead to a small decrease in energy (around or 
smaller than 2%) without any additional relevant effect when the optical efficiency of the 
fluorescence telescope or the atmospheric transmission is included in the calculation. On 
the contrary, using the K-B FY would give rise to a significant energy increase of nearly 
20%. However, as anticipated in the previous section, this deviation is reduced down to 
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Table 1: Percentage differences (1005s) between the primary energy reconstructed using either K-B or 
Nagano instead of the N-A FY data set, for typical GH profiles of p and iron 10 20 eV showers. These 
results have been obtained using the atmospheric profile of December. The effect of the filter F x , the 
total optical efficiency E\ and the atmospheric transmission T x assuming shower-telescope distances of 10 
and 30 km is shown. Uncertainties in IOOSe are estimated in about ±0.5 and thus null results represent 
energy deviations below 0.5% 
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Table 2: Same as table Q for 10 19 eV. 



about 2% when the optical efficiency of other components are included as pointed out 
by [24]. In particular the filter reduces this nearly 20% energy increase down to about 
7%, with the optical efficiency of the remaining components of the telescope responsible 
for the further decrease down to that final 2%. The effect of the atmospheric transmission 
is basically negligible 

In regard to X max reconstruction, using the FY data sets of Nagano or K-B would 
decrease the shower maximum depth by about 1 gem -2 (see tables [3] and [4J. The effect 
of the atmospheric transmission is nearly negligible. These results both of reconstructed 
energy and maximum depth are basically independent on primary energy and shower 
angle. As expected, they are also nearly independent of season. 

The effect of the humidity and the T-dependent collisional cross section on the N-A 
data set has also been studied. As already mentioned these effects vary with geometry 
and season due to the different atmospheric conditions undergone by the shower track, as 
well as with the nature of the primary (mass and energy) due to the different longitudinal 
development. 

When the water vapour quenching is taken into account, large deviations in the 
calorimetric energy (up to 7%) are found for humid months (e.g. February) and shower 
maximum close to ground (i.e., either high energy, light primary, vertical incidence or a 
combination of them). This deviation lowers down to about 1% at 60°. The effect of a 
collisional cross-section dependent on temperature ranges from 6.5% for inclined showers 
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K-B/N-A 


Nagano/N-A 




30° 


60° 


30° 


60° 




P 


Fe 


P 


Fe 


P 


Fe 


P 


Fe 


Yx(X) 


-1 


-1 


-2 


-2 


-1 


-1 


-1 


-1 


Y X (X)-F X 


-1 


-1 


-2 


-2 


-1 


-1 


-1 


-1 


Y x {X)-e x 


-1 


-1 


-2 


-1 








-1 


-1 


Y x (X)-e x -T x (X) (10km) 








-1 


-1 








-1 


-1 


Y x {X)-e x -T x (X) (30km) 


1 





-2 


-2 








-1 


-1 



Table 3: AX max differences (gem -2 ) between the reconstructed values using K-B or Nagano instead 
the N-A data set, for typical GH profiles of p and iron 10 20 eV showers. Uncertainties in AJf max are 
estimated in about ±0.5 gem -2 . Null results represent absolute deviations below 0.5gcm — 2 . More 
details on first column are given in table [T] 





K-B/N-A 


Nagano/N-A 




30° 


60° 


30° 


60° 




P 


Fe 


P 


Fe 


P 


Fe 


P 


Fe 


Y X (X) 


-1 


-1 


-2 


-2 


-1 


-1 


-1 


1 


Y X (X) ■ F x 


-1 


-1 


-2 


-2 


-1 


-1 


-1 


-1 


Y x (X)-e x 


-1 


-1 


-2 


-2 





-1 


-1 


-1 


Y X (X) ■ e x ■ T\(X) (10km) 





-1 


-1 


-1 





-1 


-1 


-1 


Y X (X) ■ e x ■ T X (X) (30km) 








-2 


-2 








-1 


-1 



Table 4: Same as tableHfor 10 19 eV. 



to about 2% for vertical ones. The total effect in the primary energy is displayed for all 
months in figure [7J Excluding the February - May period, it ranges typically between 5.5 
and 6.5%. Notice the significant variation of 6e with season for vertical showers, which 
is due to the humidity effect. 

The correction in shower maximum depth due to the temperature effect is AA max w-5 
gem -2 , nearly independent on the shower and atmospheric features while the correction 
for the humidity effect ranges from about 8 gem -2 for vertical showers in austral sum- 
mer down to about 1 gem -2 at 60°. The total X max deviations including both effects 
are displayed for all months in figure [8] As can be seen in this figure, the corrections 
for inclined showers are negative and nearly season independent. This behaviour was 
expected since the slope of the function FY ratio versus depth is positive for the T effect 
(see figure [3]) and, as already mentioned, the temperature profile is nearly constant along 
the year. On the contrary the correction for vertical showers is strongly dependent on 
the month again due to the humidity effect. 

The experimental analysis of the fluorescence quenching due to water vapor as well 
as that of the temperature dependence of the quenching cross section is rather difficult. 
As a consequence these measurements are very scarce and subject to large uncertainties. 
The procedure presented here can be used for the evaluation of the effects of the errors 
in the P^ and a parameters on the total uncertainty of the reconstructed values of the 
shower energy and A max . For this purpose we have recalculated SE and AA max after 
increasing (decreasing) a x by a factor 1.5 (0.5) and increasing (decreasing) P^ by a factor 
1.2 (0.8) for all molecular bands. In general the effect of such large eventual deviations 
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Figure 7: Percentage increase in reconstructed Eq value when including humidity and a collisional cross 
section dependent on temperature to the N-A data set, versus month. Results are shown for both protons 
and Fe nuclei of 10 19 eV (triangles) and 10 20 eV (circles) energies. 
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Figure 8: Increase of shower maximum depth due to humidity and a collisional cross section dependent 
on temperature to the N-A data set, versus month. See figure [7] for more details. 
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in these parameters is very small. The largest effect is found for high humidity months 
and/or showers more sensitive to T and humidity effects. 

For instance for 30° protons of 10 20 eV the 8% increase in energy for the February 
month would move within the 6% to 10% interval while the corresponding shift in max- 
imum depth of AY max = 9 gcm~ 2 remains unchanged. For the same kind of shower 
(p, 30°, 10 20 eV) the 5% energy increase in the August month would be within the 3 to 
6% interval and the negligible X max effect reported in fig. [8] would be in between -2 and 
-1 gem -2 . Finally for Fe showers of 60° and 10 20 eV the 6% energy increase in August 
would be in between 3 and 9% and the corresponding AX max = -4 gcm~ 2 value would 
be within the -5 to -1 gcm~ 2 interval. Note that the above examples represent the cases 
with the largest dependence on possible uncertainties in the P^ and a parameters. 

6. Conclusions 

A simple analytical method has been proposed to quantify the influence of the as- 
sumed FY on the shower reconstruction, in particular on primary energy and shower 
maximum depth. The longitudinal development of the shower is described by a Gaisser- 
Hillas profile which is slightly modified by a variation in the FY assumption. Several data 
sets of FY including absolute values, wavelength spectra, as well as pressure, temperature 
and humidity dependencies have been used for this study. 

Using this simple procedure we have confirmed that the Nagano- AIRFLY, Nagano and 
Kakimoto-Bunner data sets lead to close values (within around a 2%) of the reconstructed 
energy, as long as the optical efficiency of the Auger telescopes is taken into account. The 
corresponding effects on Y max are typically smaller than 1 gcm~ 2 . 

The dependence of the FY with atmospheric properties (P, T and humidity) and its 
effect on reconstructed parameters of the shower have also been analyzed. These effects, 
when combined, introduce typical deviations at the level of 6% in the reconstructed 
primary energy. However significantly larger deviations can be found for vertical showers 
(30°) in humid months. The deviations induced in X max by temperature and humidity 
are opposite and they nearly cancel when combined. Deviations of about -4 gcm~ 2 are 
found for inclined showers (60°) while for vertical ones the effect is strongly dependent 
on humidity with large positive deviations in some months. 

These deviations in primary energy and X max values are not negligible when com- 
pared with typical uncertainties (both systematic and statistical) in the reconstruction of 
the shower parameters. Notice, however, that they can be easily corrected as far as the 
temperature and humidity effects are included in the reconstruction algorithms. On the 
other hand, we have found that even relatively large uncertainties in the quenching pa- 
rameters associated to humidity or the T-dependence of the collisional cross section have 
not a significant impact on the reconstructed parameters. In fact the derived systematic 
uncertainties are below 2% for primary energy and below 2gcm~ 2 for X max , and there- 
fore they represent a small contribution to the total uncertainty in the reconstruction of 
the shower parameters. 

The results shown here have been obtained from the whole longitudinal development 
of the fluorescence light. In a real case the reconstruction of the longitudinal development 
is restricted to an interval given by the field of view of the telescope. In addition, the 
contribution of Cherenkov light is a very important ingredient in the reconstruction of 
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real data [55] not used in our simple procedure. These simplifications could give rise to 
small discrepancies between the predictions of our simple algorithm and those from the 
analysis of real data. Nevertheless our results are basically in agreement with those of 
[2] obtained from a detailed reconstruction of a sample of MC events. 
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